Papers with reading comprehension and translation capabilities
FilBench: Can LLMs Understand and Generate Filipino? (2025.emnlp-main)
Copied to clipboard
Lester James Validad Miranda, Elyanah Aco, Conner G. Manuel, Jan Christian Blaise Cruz, Joseph Marvin Imperial
| Challenge: | Despite impressive performance of LLMs on English-based tasks, little is known about their capabilities in specific languages such as Filipino. |
| Approach: | They propose a benchmark to evaluate LLMs across a diverse set of tasks and capabilities in Filipino, Tagalog, and Cebuano. |
| Outcome: | The proposed benchmark reflects the priorities and trends of research in the Philippines . it finds that several LLMs suffer from reading comprehension and translation capabilities . |